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(54) METHOD AND DEVICE FOR GLOBAL ACCELERATION LEARNING FOR NEURAL CIRCUIT NETWORK 
MODEL 

(57)Abstract: 

PROBLEM TO BE SOLVED: To shorten the search time by repeatedly 
executing a global search when a solution having an evaluation function 
value smaller than the evaluation function value of local solution can 
not be searched. 

SOLUTION: A correction amount calculation part 207 calculates the 
correction amount of weight from the difference of weight or gradient 
transferred according to the command of correction judge part 206, a 
weight correction part 208 corrects the value of weight, and an 
extremal value search convergence judging part 209 judges whether 
the stop preference of extremal value search is satisfied or not. A 
global search part 210 executes minimization to an evaluation function 
and searches a point where the evaluation function value gets smaller. 
A global search convergence judging part 21 1 judges whether the point 
found as a result of global search satisfies the stop recurrence of 
macro optimization on not When the point of smaller evaluation 
function value is found out, a limit condition coefficient change part 
212 changes the coefficient of limit condition. When the point of 
smaller evaluation function value is not found out, a global optimization 
coefficient change part 213 updates the coefficient of global 
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optimization and when each processing is not finished even after repetition, processing is finished. 
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i ^vmtmm^rcmmmm 2 0^77 fx 

su!2H 3 ©Xxy -?X$tb tckKms ©£iEfc «ktf k — 

1 fcmvmimmz&mmcmmm turn* 

fClSTf^!! 4 ©Xt -y 

ButBIg 2 fe<fc tf|g 4 ©Xt -y 7T*:}<i6/c£|Efc<fct/&l 

ffiSUKftgU C©iiIE!tfC k - 1 gfg@©!&<D|gIE 
lt«tt**Slli:/c:fiEIl*Sl*KfligijK:llD^5g5 

mm, 20Xf7 fXXtbtc k g|gg©£|Efc k - 1 g 

aao^E*»&43E<oa» (ks«sfcfect5^na©n 

Mi2H6©7r7/T*»fekSSg0^iB<DM^i: 1 
KMi©k- l£fiB©&JE©ig#©rtStefWU C 

l*C«iElt. gfckj£tgB©£)E©M#tk£tB 
©i*©§jEt©rtii<ii£ k 5«a0fce©8#4>** 
£ <//l/v) ©2«fcj;tf2-eiiJ*bfc8S (X*5- 

7©Xr-y7t, 

ksigcD^Et k- 1 zmwrnrnftitmitc 

buKU 5 7 ©Xr-y 7T')£#>/cl#©§ It£ 

ffl^Tl#«S**icffl9Jlt:Hfr-r 9 ©Xf >y 7 



If M¥ 10-334070 

2 / 

R»Off±»P*i(lfct$T|(IEa24v>Lai 0©X 

x, mMicmz>mmM®L<Dmmzmmwzmc% 

frfcBW*/<7;<-**||BLfc& miieBg2©XT7 
12151 l&t^Ugl 3©Xr7 7©*is«ijs£agg 

m±mmrc-$%^®etitmm7tz% 1 3© 

[W**2] #8[H8S^tT;l/©l*©S]^, 

sw©i!B$, ™t»8s, fitted 

a£-^*33?l ©Xr'y^fc, 
^Wt^Aftr-*© 1 ^*-y*tt«BgMBtr;l/ 

$JE£fW1- 2 ©Xt 7 7i , 
SafckESBOSlElcSU, coMti&m (k- 

1 sag fc-r «) ©^fta^ic JFjnta«a*s ufc 

lining kS«lci3tts3p}Bfka»fi%§gj({sic 
9ui2M! 3 ©Xr -y fX'^tc k g^g ©tiEfci t>* k - 

1 mmm<mmzmmmcmmm\:mw 

^S$fc^s^©tgjn^inxT^ 

BufaH 2 XS m 4 ©Xr -y ^T'^46/c£lEte «t D'^l 
*C»JELfeffl9JO^S*%3lli;fe«iEa*»S*Slc 
(IS'JtH-gCt, C©jgEltc k - 1 5Sg©{itt5{c1i 

•y/t, 

B«iEll2©XT-y/T**«)fckSaa©&iEi:k-ig 
aa<043E*»543BO«» (kg^afCj3lt§^)E©M 



(3) 



fflBM 3 ©Xt7 TVJM>ltksmsniSKt k - 1 e 
***Ufc»iEl*«SIH|icjB»Jft:«»*icta**fr 

®«^tt»Eff»*«tat5S 8 <DXt y ft, 
i> 

IT, £l lO^rv^SHfu fffBE*<Dffjt£ip 

fi8$Of? JtSip*»fe-r *T«H3S 2 4i> LH 1 0 to 
^ry^sanfrf 5 111 0^T7/i 
S«lia>)g^nfc^WM«fMt L 20 

»ffi§BS*«js u c <dw mm&zm'iMtt s c t tc £ 

^WW>«BIW«IWf t*» l 2 cxf7 

zztzmmmvis^itei^ ±mm®{tmt so 

JEST 1 1 4^ LSI 1 3 <DXf y 7<D±Wm&ZBm 

&B£Mtz¥mmm*mmm®micmL 40 
fi«§o¥»ft«»fej:tf k- iRmsiommm 

micmu zmmmvuizici}, ne«3o so 



If iff 1 0-3 3 4 07 0 
4 

fctt kEfigfcjtf k - 1 R&BWmat&VMSm 

tetan 8 «xf7 yti, k Easowt^t k - 1 

KkEtlg^EfcsH;, c<7)®£iESEM (k-i£ 
JuSU kESBfc;fef*5¥fcfca*ra*ittM-53l3 

MtE£3<Dm?mt: kmsm&tszv k - l E 
aB^^JSfta^fi^^snsicfiaitsiiufcgangs 

WES 2 feiQ'H 4 <0¥ST'^i6^Efe c kt;§a* fc 

Lfc«Wo*B***i;ft«EiEl*««|M5ic«ij 
Ktt* L, C OHiEllc k - l E^gcDl^fiDgiElfc 

St, 

itiilESg 2 C¥gT*i6ft k E«g<D^iB t k - l E^g 

**jfiitoii£K: it, gijissr 5 o¥a-p*»fta*og 

iE«t> ^kE^g<D4jE<DM^tkESgfl[)l*0D 
^iEl<Ort«f|^kEagcD^iE0[)g^<7)^tj (y ;l/ 
^) ®2**J:tf2?|»ftLfc8ft U*7-t) tk 

fettidfiS 4 ©?a-p*«>fc40Ei:3Fitfl:a»o5||»jg 
**«iEfl<0«^K J4«iEl%(*ip 7 O^gfc 

kgfjBo^Ei k - , RmwmtmzmLk 



(4) 

5 

MES 5 fc<fc O'S 7 Of ir?;R#fc«#0{iIEt%ffii> 

i oo^atjcsfta^sa^ff-rs^i oof at, 
fammt&imztirz&T'&zm&zwmmtL io 

T, 13r»Ptte^5i¥ffiBilS(Dfiiffl^i3^l/>/j:l»f/c=S: 
ft Hft*«|j£ U C OffttBBa*«/Wkt 5 C t fc J: 
•3 9 S 1 1 Of® fc , 

$%iHRRft«««r«iKm{T*«^«dB t mm 

z%tmmmT<2%\,^^ mmmmmt 20 
aires 1 i£^lsi sofat^*^^* 

*mmn, ¥m<omm, ¥»{k«m wit* 

*P<8> ffittfettaftK, «J:tfAMftKi{bicBn-«ft 30 
*ST^* TJdt-*© l ^^-yittJSHKm^T 9 ^ 
U COllJMt 1 ><*->0lMfT-*fr5if-Jf £ ft 

l fre»¥?i fk««»*5lv^cffl*tWBS 2 Of-gT-;RJ6 
fckE«B©$jBfcjRi; x £©<I£ (k-ig 

inies 3 omvmtc k gagosiEfc it; k - 1 r 

frt5S4©fSi; x 

MIBS 2 J: tfS 4 Of &T*#i&fc43E:& J: tf #!#ic 
«*bft«9Jo*B**«i;ftl!lFjEKSBi6*lcfflgiJ 50 



ffBB¥l 0-3 3 4 0 7 0 
6 

fcftH U COUEiElK: k - 1 gtftg (Dfltftgicflttt* 
*ft Cfe«jEt**a*t flfflKipit *S 5 Of s t . 

O^EA^^JEOM^ (k£agfcfctt*£jE©M#£ 

tuIBS30f|§T'*46fckgago£lEt k - 1S«B 

nfcti, iJEg50#ST*»fc«*o«iElK:, lift 
OS«^fc fctts&iEOltffc H£fi<DttiEftft$£ C 

ias 4 <»mT-#&m&t¥%itm<ommim 

Ol^fC (iill^ftiiD t ftl^S 7 Of S fc , 
kSSgO^Et k - 1 E«B©¥»{"fc«£*S*Lfc 

ov^ffrasftmrrss 8 of mt, 

MfBS 5 fcj: tfS 7 Of-&T*:£a6fc!#0«IEl£ffl^ 

lt, si i (D^mczzmmnu mrnmm 
i^lsi o<o^aKj:«ftaftga*ffr5Si oof 

^1£T\ J: •) felled 

«4iwsH»fl*fr*»A^«Ttfc«^tt % mm 
GKRtz mm vm. mibs 2 ©f^gtc 

«k*ffll«»<oa«««4*fft*Si 20fat, 

*«a*«BLfea, Miasi lof-gtcfc^iaatci 

tuIBS 1 lftV^LSl 30fig£J;5*Jgtfjfi5fis£ 

imm6i Buiessofati, ifrzmimm 
*^m*v>-\Rmwm{tmm\cm. co 

sSIES 4 Of g(4 % MKS 3 Of-gT^/c k 5« g O 

¥» f«i»*j:if k - 1 Rmommmzmm 

^S^oi^iiD^^in^T^g^^iiJjp^^, smsjmi 



tuE^ 7 (D^mt, Mian 3 <D$&L?#ibrc k g o 
¥i m% t k - 1 msmmtmz&mmcm 

)sxrs k - 1 mmm\tmv>m&mm<DWi 
m%8<omtt, kKmwmtmt k- 1 

[00 0 1] 

^ffl^/c^a, imsi^s ^fflt,>/c5aa% 2 taua 

LT^mW^jNffi^EJjg U Aigf^Hf/JMiKcgtiSt l ? s 

y - 8 i s (x) — i 



1-exp (- £ 
i«l 



(5) »S¥ 1 0-3 3 4 0 7 0 
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[0002] 

[0003] mmmmmtTm, 02£*tj: 

K I*. ;V7*fr£}M£ft5o 3.-7 Mi, Ml 
<Da-7bOtil7]fi (x, (i=l, 2, L, L: 
buS03.-7 (w, , i :S#©#*t) © 

ffl©l£fck te^aV^JX (bi , i :a--yh©S 

mm (f (•) ) mhm ( y ) ^ttj^u 

ffl;WI£:ftM©a-y h^&mtzm&*t>0 (H2, 
^ (1) ) o IU CCT'tt, \t)M<D3.-y h(DAlli 

20 jgg&H& f ( • ) mMT'3bZis?*<< mm 

[000 4] 

[&1] 

I 

... (J) 



W\1 



m&mzm'b<ttz>£o%nj*<D®m%.tz>rcib mx&mmT-* cr, (j = i, 2, -, m) ) <m\ 



mm^m^iKotam co, 0 = 1,2, •-. 



j-1 



(3(2)) £ffl^3 0 
[000 5] 
[|S[2] 

• (2) 



D.E.Rumelhart 5>tf #&@g8$^r;PcD^g7£fc UTS 

/ewe mwmzwmMick outs* 



Jl^L/c &ffi : %%mfc, Ffflft;£, : "1 

ilfb-Kick Outi£-" , Vol.ll3-C,N 
40 O.12,pp.ll54-1162(1993)#!«0 0 

[0 0 0 6] £<D¥mmTlC*t a 

[000 7] 

[S3] 



(6) 
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Kick Ou tffi 
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Aw k -diag (* k ) g fc +aAw tl 



(3) 



DK4] 



Aw v 



-d i a g (* k ) g k +oAw H 



Aw, 



if y k y k-l > 0 



yl Aw, 

-dlag(» k > g k +«Aw w - — ±— |-y k 



(4) 



2lly k |J ^ 



if y{ y k-1 < o 



•fci 












- "k-l.i ■* 




• 8 k.i < 0 


... ( 5 ) 






otherwise 
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L*i/ft*«6, Kick Mmz-mummmm&T' * 

[0008] mfm^iB±icjwrms<ivji«<» 
[ooo9] cft*»at**iijfcUT, swsrk: 

m i n f (w) 

sub. gj (w) £ 0 1=1.2, 



&mur ( o) s) *«iku cntg^-airfc 

ftfthfifcHURf 5 2V 1 5 y * b y*)l7)ldV XL 
: Y.Yao:"Dynamic tunneling algorithm for glo 
bal optimization", IEEE Trans. on Sys.Man and Cyber 
n . SMC- 1 9-5 . pp . 1 222- 1 230 ( 1 989) #M) iffl£L2tlT\<* 

o zmmmmmz. *> Hums* 

[fl-2] 



(7) 1 0-3 3 4 0 7 0 



rooio] * # [R5] 
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dw i *f ■ * 



ALU 

f a if 8 > 0 

f * (a) « 



(V) 



(8) 



I 0 if a < 0 
tOO 1 1] * m 

»<*w opti *<H (w) ose^iVi+^^ i& tt«c 
[00l2] , _ ★ ★ DR6] 

H (w) = _cp^_ 

A • B • • • M 



- 0 



(9) 



au A«[(w-w optl ) T ( v -w 0Jrtl )]^ 



[0013 1 A * W4] 

COffftTMi, Stt«*lcJ:*je3rj|Pw**<«i6&n*^ ±* ((io)^ 

[l * 7] - ♦W53 

f (w) = f (w) - f (w*) ^ 
(w) ttMM^St. 

f° 0l4 3 _ * * QK8] 

_ ■ , * 9* fir) 

Kw-w ) (w-w >] J«l J 1 »w 

k f * (7 (1=1. 2 . -.a) .-a© 
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[0016] tit. mmtcit. *«wasR*ffdi«K 

[00 17] 

B8iH:MfifebJ:3i:**Ml] Kick Outtttt, ffffiM 
5ffifi«^S07;UJyXi»T?fe»), Kick Outft<D*-e 10 

[0 0 18] Jfc, ^t2«y^h^y>^7;U^'J 
XAfi, *«Wft«/hfi*a§gt«7;l/ J'J XAT'353 

[0 0 19] HfC, Kick Outfit ^gt^f^^-y 

t\ -g^^^oKick outm%micm%.¥%mcmm 

LTfe, Kick 0utft£H8g©tajB»attff6tl&^fc 
[0 0 2 0] *»fl8tt, ±!2k&#T&£ftfcfe©T\ 

fitfSftLTt^iswfta/j^agR-rs c ttf-ets 30 

[0 0 2 1] 

[RE*l!H*-r3fc«><0?a] ±EB»*afiJtf5fc 
Ml£fg<D{#itSm Mi&ffOftft, *J:tf**tt« 

*-y\zmmmmmLm^Tmi<o&m& (k 

SttSO^GlciRi;, cofiltifiSi) (k-isai 
U k5a§fcfcit5¥rifk^<i*fH[-r5^3<DX 50 



ftHTl 0-33 4 07 0 
14 

k - 1 KBSOWtflfttiMteMyttSlcflBlK: 

S**fl9JfcJWf*«S4 0Xrv7i:, H&fEif 2 fcj; 

fl8>lo¥9*«3ei;^|)EjEl««fijK«icfl9Jic|f|| 

%*Ufe*iEfi*fta*tfljgiJtiiniL«g5©^ryy 
^ «WaS207T'yy7**fckKag<D^El:k- 
lE&g<D$JE*>e>43E©!§# (kESBlcfcttSdgB 
OH^fc-rs) *ttSf5^6(0^r->^t, iuIBSg6 
0*ry:/T*»fckEgB©4gE<c>!g#i: iRffliQ 

Elfc, H£kSSB^E©l§#£ kgfiB©!*© 

*iEaortafi*ksai«o^Eoii»©**« 

V) 02|fcJ;If2T'iILfc8S (X*7-i) k 
yXt, kEaB0ftE*k-iE«B©¥»fl:l*#* 

& MEIgl KOXryXfcK^ ituESgl iftv^bS 

1 3©XT7Xo^ws^ganffL, ti^^n/c 

[0 0 2 2] ||^« i E«o*IBaic*oTtt. IFfflS 
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[0023] sfc, imm2Em<D*mit, mm 
m, mmm. mm, miEmmm, mm 

m& \^tzmmE% 2 <d*t 7 -fx^tz k m. b 
kmicwzw-mtmmz&mmmcmtzn 3 

OZy-yft, MfB^3<DXr •y^T-Sci6/ckgagO 
fclBBfcitfk- l S^g^¥?ifb^l%^^(c<i 

^g$wijt^if-r5^4©XT7yt, sue 

fg 2 fc J; tf |g 4 0Xf7 ^T*^*6fc^E*5 J; 
tcftlU c<D<f life k - 1 g'<tg<DWlJgtcf!1£3s 

f512£2©Xr7:/T*jR&fckK«gcD^S2£k- 
15«B©^ffifr££)gE©ii# (kg&BKfctt^E 

©^T7/T'*i6fckS^g(DfilEi: k - 1 5^B<0¥ 

£SIt££ljg«(cfi8iJ(c£lwc;!)p;t 5 frt urn 
iEffio^fc«Jiii^#inL^i^ 7 (DXT7 -ft, 

v&mmmt k - 1 k^bwi itmmwbtc 

5 ts&lf% 7 <DXr 7 7T'S<i6fcfi#a)itIEl£ffll,>T 
l*^g^S»c«tHfr1-S^9<D7.T7yi:, ffiffi 

X, %l 10Xr7 7*£HfrU fiK@ggs<D#ibS?p£ 
$/c££^l§£K«:, Hljl2S2£0XT7y{clf), S@ 
StOffltg^gS/c-f £ T'tuiam 2 £v> 1 0 <0X 
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mti§£imji. mzt£%^i§£x\ i?imm\z 
mmmx'trz^m, ®mmc®tz>^yt-** 

IS 1 fcfc, iwESg 2 <DXt 7 ^WfDlfflgl^^^f 
TSUI 2cDXr77 p t, ^me?tfc^-5ff<iBiMMJ; 

1 KDXT-yyicm*), MEHl l&l^Lgll 3©Xr 

immmjtz>%\ 3<D7>TV?tz%tz><it*m 

[0 0 2 4] fS^2l2«<D*^t^oT«, IfffiM 

*%£m^rcw%¥W£}:\ct5^x¥m?$<DMm 
[0025] m*m3E®(D*miit, mxm 1 
»fk»ff«%3i^fcfi% k - 1 Kmewmmimt 

IWt, BufES4<DXr7^ Mie^3£DXr7 7'T* 

s»fc k sagoTOkffl^feiQ* k - 1 msom 
ks«i©JFi»{k«»i: k - 1 Kmwm 

mZ%3(DZTy7X'$.iibrcMb<D®Emc, kg 
»Bfc*tt5^EOS^lCRra<!)»iE«a**i;fcj|iE 

hues 8oxf7 y*^ k K'mmmmftt k - 1 

[0 0 2 6] «*«3E«0*fK8»cSoTtt, IfffiH 
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[0027] mmm<D*mit, mmmr 
;KDi#©8HBii, ^aaoaaa* ^aaoaa* 
*mmm, am mmmma, aaa*& 

mmm^mmm z <om-?mrc k m 
mmmc cmtxmm a-imstt 

k s« g tc fent § ¥«tfb$sMi*fnn- 5 a 3 <om 

t , MEa 3 ©#aT*«>fe k ga g <D£|Efc J: tf k - 

mmtQt^ a*aa#aa<o«^ ^gat^a 

CT2B**«'J>;£tf, &?a*$asij 20 
KSfhr<5gl4<D3M£^ ME%21S&lf%4CD^&T' 

fc£ IEl*#SJ|«tfflgiJjcH-g U C mJEMlc k - 

fc ksaa<o^Efc k - 1 zmsmm^mamft 
ammcmzqmvmttz) *aa*aa6 

fit i mmk- 1 &m<D£im(Dm<Dfimmn 
u cojistftfl^KH;, aeaso^a-Faa 30 

fci&cDglEtfc, ItkS^g£D^EOl^i:kSS 

-i) fckgaa©43Eoa»**i:feifiEi*i*t 

#0*Sa*#Ea««£fc tt«IEa*fliinL4^a 7 
<^gt, kSagodgEk k - 1 sago^if fbS8» 

*aaucaajWEa0«sttsiEaa*»'>u us 

fiuEIg 5 fcitfjg 7 0^aT»»fea*0»El*ffl^ 40 
Tfi*£g*«fca»Jfcaa?-*a9<i!>^gi:, ft£a 

p^i/cti^ca, aasa**7LT» si 10 
ftt^tettt. s&eai 2 «¥s»c * « MicR 5 % ae 

aaoffit***«fe T $ T'BuEil 2 4 ^ LSg l 0 <0¥ 

sicj:«ffls%Kajiff-r*ai o a<» 

KJ:5ffi«*nfcjS-P*SJiBi<g%ffllBfiIi:UT, Jiff? 

acfettsaaaaaaa^a ►> m^tcmcmmm 

*aau £©ffaaS*8'M:-r5c:i:fcJ:»>*8» 50 
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a**ff$»i lo^at, ttmaatoaii**** 
fc**att«7u mrzz%\,^T\ mam** 
vmmmmi. 9 fcaM^aaaaaa***** 

sum aEazo^atcj^aagtaoaaaa* 
afrtsai zm&ts mm\cmmmm& 
*) feat'h*&ffffiH»a**'r*a* t »a-p*4v^a 

1 lo^aicisaaKat), mmi i&^ugi 3 
o?aicj:s*««asR*sa^fTL, aasnfcaa 

^summit z% 1 3<D^at^wr-5ct^ 

[0 0 2 8] a*«4E«0*|»|Blc*oTtt, ffffiM 
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